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Abstract 

We consider the zeta and Mobius functions of a partial order on integer compositions 
first studied by Bergeron, Bousquet-Melou, and Dulucq. The Mobius function of this 
poset was determined by Sagan and Vatter. We prove rationality of various formal 
power series in noncommuting variables whose coefficients are evaluations of the zeta 
function £ and the Mobius function fj,. The proofs are either directly from the definitions 
or by constructing finite-state automata. 

We also obtain explicit expressions for generating functions obtained by specializing 
the variables to commutative ones. We reprove Sagan and Vatter's formula for fx using 
this machinery. These results are closely related to those of Bjorner and Reutenauer 
about subword order, and we discuss a common generalization. 

1 Introduction 

Let A be an arbitrary set and consider the free monoid, A*, of all words over A: 

A* = {w = w(l)w(2) . . . w(n) | n > and w(i) G A for all i}. 
*This work was partially done while the author was on leave at the Institut Mittag-Lefflcr and at DIMACS 
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We let £{w) denote the length (number of elements) of w. 

If P is the positive integers, then P* is just the set of integer compositions (ordered 
partitions). We put a partial order on P* by saying that u < w if and only if w contains a 
subword w{ii)w{i2) ■ ■ -w(ii) where I = £{u) and 

u(j) — w (h) f° r 1 — j — I- 

To illustrate, 334 < 34261 as can be seen by considering the subword 346. Note that 
integers will be typeset in boldface when considered as elements of P*. Bergeron, Bousquet- 
Melou and Dulucq [1] initiated the study of P* by counting its saturated lower chains. This 
work was carried on by Snellman [10, 11] who also considered saturated chains in two other 
partial orders on P*. One of these posets was originally defined by Bjorner and Stanley [5] 
who showed that it has analogues of many of the properties of Young's lattice. Sagan and 
Vatter [9] determined the Mobius function of the poset we are considering. Here we will 
use generating functions over monoids to give more information about the Mobius and zeta 
functions of P* as well as rederiving the theorem of Sagan and Vatter using this machinery. 

There is a strong connection between this order on P* and subword order. Considering 
A to be arbitrary, we define subword order on A* by letting u < w if and only if there is a 
subword w{ii)w{i2) ■ ■ ■ w{ii) of length I = l(u) with 

u(j) = w{ij) for 1 < j < I. 

For example, abba < ababbbaa since w(l)w(4)w(6)w(8) = abba. Context will make it clear 
whether "<" refers to subword order or composition order. Bjorner [3] was the first to give 
a complete characterization of the Mobius function for subword order. See [9] for a history 
of this problem. In particular, Bjorner and Reutenauer [4] showed that the Mobius and zeta 
functions have rational generating functions and were able to reprove the formula for fi using 
these ideas. 

The rest of this paper is structured as follows. In the next section we provide the necessary 
definitions to state Bjorner's formula for [i in A* as well as Sagan and Vatter's result in P*, see 
Theorems 2.1 and 2.2, respectively. In Section 3 we prove the rationality of monoid generating 
functions for /i and ( on the subposet {1,2, ... ,n}* of P*. Our demonstrations are either 
based directly on the definitions or use finite-state automata. By specializing the variables, 
we obtain explicit formulas for related generating functions in Section 4. Surprisingly, results 
about hypergeometric series are needed to do some of the computations. The next section is 
devoted to another proof of the formula for [i in P* using the generating function approach. 
Sagan and Vatter showed that both Theorems 2.1 and 2.2 are special cases of a more general 
result about certain partial orders which they called generalized subword orders (and which 
have been studied in the context of well-quasi-ordering, see Kruskal [7]). In Section 6, we 
indicate which of our results can be proved in this level of generality. We end with a section 
of comments and open problems. 

2 Subword and composition order 

We will first present the formula for the Mobius function of A* in a way that will help 
motivate our definitions when we get to P*. We will not define the Mobius function itself, 
but that background can be found in the text of Stanley [14, §3.6-3.7]. 
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We begin by giving an equivalent formulation for subword order which will be useful 
when we get to /x Suppose we have a special symbol with G" A. Then the support of a 
word r) = r/(l)?7(2) . . . 77(71) G (A U 0)* is 

Supp?7 = {i I T)(i) 7^ 0}. 

An expansion of u G A* is a word 7? u G (A U 0)* such that the restriction 7] u to its support is 
u. Taking u = abba as before, then one possible expansion is 77^ = aOObObOa. An embedding 
of u into w is an expansion 77^ of u having length £(w) and satisfying 

Vu(i) — f° r all i ^ Supp7? u . 

Clearly u < w in subword order if and only if there is an embedding of u into w. In fact, 
the example 77^ above is the embedding which corresponds to the subword of w = ababbbaa 
given in the previous section. 

The Mobius function of subword order counts a particular type of embedding. Suppose 
a G A. A run of a's in w is a maximal interval of indices [r, t] such that 

w(r) = w(r + 1) = . . . = w(t) = a. 

Continuing with our example, w = ababbbaa has runs [1, 1], [2, 2], [3, 3], [4, 6], and [7, 8]. An 
embedding 77^ into w is normal if, for every a G A and every run [r, t] of a's, we have 

(r, t] C Supp 77 u 

for the half-open interval (r,t\. In our running example, this means that the cVs in positions 
5 and 6 as well as the a in position 8 must be in any normal embedding. (Runs of one 
element impose no restriction since if r = t then (r, t] = 0.) So in this case there are exactly 
two normal embeddings 77 u into w, namely 

Vu = aOOObbOa and OOaObbOa. 

Let (^) denote the number of normal embeddings of u into w. 

Theorem 2.1 (Bjorner [3]). Ifu,w G A* then 

Mu,^) = (-i) H " H Q • □ 

Putting everything together in our example, we obtain 

/j(abba, ababbbaa) = (— l) 8-4 -2 = 2. 

In P*, the definitions of support and expansion are the same as in A*. However, the 
definition of embedding must be changed to reflect the different partial order. In this case, 
define an embedding of u into w as an expansion 77^ such that ^(77^) = £(w) and 

ri u {i) < w(i) for 1 < i < £(w). 

As before, u < w in IP* if and only if there exists an embedding of u into w. 
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Of particular interest to us will be the rightmost embedding. Suppose u < w. The 
rightmost embedding p u into w is the one such that for any other embedding rj u into w we 
have Supp(p u ) > Supp^). (If S — {i\ < ■ ■ ■ < i m } and S' = {i[ < ■ ■ ■ < i' m } then S > S' 
means ij > ij for 1 < j < m.) 

The definition of a run is again the same in P* as it was in A*. So we call an embedding 
f] u into w normal if it satisfies the following two criteria. 

1. For 1 < i < £(w), we have t] u (i) = w(i), w(i) — 1, or 0. 

2. For all k > 1 and every run [r, t] of k s in w, we have 

(a) (r, t] C Supp?7 u if k = 1, 

(b) r G Supp?7 u if k > 2. 

Note that in P* a normal embedding can have three possible values at each position instead of 
the two permitted in A*. Also note that the run condition for ones is the same as in A*, while 
that condition for integers greater than one is complementary. For example, if u = 21113 
and w = 2211133, then there are two normal embeddings, namely rj u = 2101130 and 
2011130. Also, 2001113 and 0211130 are not normal since they violate conditions (1) 
and (2), respectively. 

Another difference between A* and P* is that in the former the sign of an embedding 
only depends on the length difference, while in the latter it depends on the embedding itself. 
If f] u into w is normal then define its defect to be 

d(r)u) = #{z | r} u (i) = w(i) - 1}. 

The formula for the Mobius function of P* is as follows. 

Theorem 2.2 (Sagan and Vatter [9]). Ifu,w £ P* then 

where the sum is over all normal embeddings t] u into w. m 

Finishing off the example of the previous paragraph, 

/i(21113, 2211133) = (-1) 2 + (-1)° = 2. 

Although this example does not show it, it is possible to have cancellation among the terms 
in the sum for fi. 

3 Rationality 

Let e denote the empty word in A*. For this section and the next one we will assume that 
A is a finite set. Let Z((A)) be the algebra of formal power series in the noncommuting 
variables A with integer coefficients. So every / £ Z{{A)) has the form 

/ = ^2 c w w 
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where w & A* and c w G Z. If / has no constant term, i.e., c e = 0, then define 

r = c+/+/ a +/ 8 + ... = (c-/r i . (i) 

(One needs the restriction on / to make sure that the sum is well defined as a formal power 
series.) We say / is rational if it can constructed from a finite set of monomials using 
a finite number of applications of the algebra operations and the star operation. For more 
information about rational series, see the books of Eilenberg [6] or Berstel and Reutenauer [2]. 
We will show in this section that various series related to the Mobius and zeta functions are 
rational. 

It will be convenient to define [n] = [l, n]. We will also use such interval notations with 
elements of P* in the obvious way. So, for example, 

[k, n] = {k, k + 1, . . . , n}. 

Consider [n]* as a subposet of P*. Given u G [n]*, we have the associated formal series 

Z(u) = Y l ™ = 'E,C(u,w)w (2) 

w>u w 

where ( is the zeta function of [n]*. We also wish to consider 

m h = E ( E(- 1 ) d( " u) ) w ( 3 ) 

w>u \ r] u / 

where the inner sum is over all normal embeddings r\ u into w. Note that if we assume Theo- 
rem 2.2 then M{u) = J2 W l J >{ u i w)w, but we will not need this fact to do our computations. 
Indeed, in Section 5 we will use the displayed definitions of Z{u) and M(u) above to reprove 
Theorem 2.2. 

The crucial observation underlying our method is that Z(u) and M(u) can be expressed 
in terms of simpler series. To define these series, it will help to have a bit more notation. 
If S C [n]* then we will also let S stand for the generating function ^2 w( z S w. Context will 
make it clear which interpretation is meant. If S is empty then the corresponding generating 
function is the zero series. If / is a series without constant term then we let 

f + = f + f + f + --- = r-e. 

Note that / + is rational if / is. Finally, a function F : [n\* — > Z(([n])) is called multiplicative 
if for any u G [n]* we have 

F(u) = F(u(l))F(u(2))---F(u(l)) 

where / = £(u). 

Now define two multiplicative functions from [n]* to Z(([n])) by setting, for all k G [n], 

z{k) = [k,n] ■ [k-1]* 
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and 



m{k) 



l-2+(e-l) iffc = l, 

(fc + - (fc + l)+) (e-1) ififc>2. 



(Note that by convention, [k — 1] = when k = 1 and + 1 = when k = n.) These are 
the building blocks for Z(u) and M(u). 

Lemma 3.1. For any u G [n]* we have 



Proof. To prove the first equation, it suffices to show that the product on the right-hand side 
produces each w > u according to the rightmost embedding p u of u into w. So such w will 
occur exactly once since the rightmost embedding is unique. Suppose k is the last element 
of u. Then z(k) is the last factor of the product. The term I chosen from [k, n] corresponds 
to the element of w greater than k in the rightmost embedding, while the product [k — 1]* 
contains all possible subwords which could appear after linw while keeping k in its rightmost 
position. Similar considerations apply to the other factors in z(u). Finally, the initial [n]* 
accounts for everything to the left of the element of w corresponding to the first element of 
u. 

The proof of the second equation is similar except that we must have a unique term for 
every normal embedding r\ u into w and each term must have sign (— Again, consider 
the last element k of u. If k = 1 then by the first normality condition, the corresponding 
element of w must be Z = 1 or Z = 2. If I — 1 then the second normality condition ensures 
that there is no element to the right of I in w and there is no contribution to the defect in 
this case. This corresponds to the initial 1 in the expression for m(l). If / = 2 then (by 
normality again) the subword of w to the right of I must consist only of 2's, possibly with a 
final 1. The factor — 2 + accounts for the string of 2's with the appropriate sign and the final 
factor of e — 1 takes care of the possibilities at the right end of w. The arguments for k > 2 
and for the initial factor of e — 1 in M(u) are along the same lines and so omitted. □ 

Note that directly from their definition, z(k) and m(k) are rational series. So, by the 
previous lemma, we have the following result. 

Theorem 3.2. For any u G [n]* , Z{u) and M{u) are rational series. □ 

We will now prove analogous results for the generating functions of ( and [i using the 
alphabet of ordered pairs [n] x [n] = [n] 2 . We could do so by modifying the arguments 
which led to the previous theorem. But for variety's sake, we will use finite-state automata. 
We write the elements of Z(([n] 2 )) as 



Given an alphabet A, a finite-state automaton is a digraph D with the following proper- 
ties. The vertex set V and directed edge (arc) set E are both finite with loops and multiarcs 



Z(u) = [n]*z(u) 



and 



M(u) = (e - l)m{u) 
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l(g>(l+2+3) 




permitted. There is a distinguished initial vertex and a distinguished final vertex denoted a 
and u, respectively. Each e £ E is assigned a monomial label /(e) £ Z((A)). 
Now given a finite walk with arcs ei, . . . , e$, we assign it the monomial 

/W=II/(e*). 

8=1 

The formal power series accepted by D is 

w 

where the sum is over all finite walks from a to to. Note that if e\, . . . , ej are all arcs from a 
vertex (3 to a vertex 7, then replacing these arcs by a single arc e = (3^ and setting 

i=l 

does not change the series accepted by D. So we will do this when constructing automata 
without further comment. We will also use algebraic operations to simplify the sum for /(e) 
if possible. 

The crucial fact which we will need is the well-known result that a series is rational if 
and only if it is accepted by some finite-state automaton D, see e.g. [2]. 

Theorem 3.3. In Z(([n] 2 }} the series 

Z® = 2^ C( u > w ) u ® w 
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and 



= / p(u, w)u <g> w 



are rational. 

Proof. For both series, we will build finite-state automata accepting them. 

The automaton D for has vertices {a, uj, j3i, . . . , f3 n }. A picture of the digraph when 
n = 3 is given in Figure 1. The vertex uj is not shown since it simply has an incoming arc, 
labeled e ® e, from every other vertex. To describe the arc set, we will consider each of the 
other vertices in turn and describe all its incoming arcs. 

If the vertex is a, then the only incoming arc is a loop labeled e <g> [n]. If it is uj, then 
we have already described the arcs into it. If the vertex is (3k for some k then there is an 
incoming arc from every vertex except uj, as well as a loop, which are labeled 



To show D accepts Z®, we need to prove that for every pair u ® w with u < w there 
is a unique way to obtain u <S> w as a monomial along some walk from a to uj, and that 
these are the only monomials in f(D). We will indicate how one can find the walk W given 
u (g) w, since then the reader should be able to fill in the details of the rest of the proof. In 
fact, we will show that W constructs w and u in its rightmost embedding p u into w in the 
following sense. If is the ith arc of W then /(e^) contains the term a <g> b where b = w(i) 
and a = p u {i) or e depending on whether p u {i) G [n] or p u (i) = 0, respectively. 

To begin, W loops % — \ times at a, where % is the smallest index with p u ii) ^ 0. (If u — e 
then let i = £(w) + 1.) The walk W finishes at uj if i = i(w) + 1, while if i < £(w) it goes to 
flk where p u (i) = k. Now W loops at /3k through arc ej_i, where j > i is the next index with 
p u (j) ^ 0. The e ® [k — 1] summand on the arc contains the necessary monomial. Then Cj 
goes from to f3i where p u (j) = I. Note that we could have k = I so that this would also 
be a loop, in which case the [k, n] summand contains the desired monomial. One continues 
in this manner until W has gone through £(w) arcs, after which it takes the arc to uj. 

The automaton for has the same vertex set as the one for Z®. See Figure 2 for the 
picture when n — 3. Again, u only has incoming arcs from the other vertices and they are 
all labeled e ® e, so it is not shown. Since the construction of this automaton and the proof 
that it does accept M® is parallel to what we did for Z®, we will content ourselves with a 
description of its arc set. Note that the interpretation of p(u,w) built into the automaton 
relies on Theorem 2.2. 

For a there are no incoming arcs and we have already described what happens for uj. 
If the vertex is /3i, then there are incoming arcs from every vertex except uj and they are 
labeled 



If the vertex is (3k for k > 2 then we have the same set of incoming arcs with labels 
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(l-e)®l 




3®3 



Figure 2: The automaton for when n = 3 



This completes the description of the automaton for M<g>. 



□ 



4 Generating functions in commuting variables 

By specialization of variables, we can get generating functions for £ and fi it terms of the 
length function £(w) or in terms of the sum of the parts, or norm, of the composition, which 
will be denoted \w\. We will also need to keep track of the type of w, t(w) = (li, l 2 , ■ ■ ■ , l n ), 
where Ik is the number of fc's in w. So J2 k Ik = £(w) and J2 k Ikk = \w\. 

Suppose x is a variable and we substitute x k for k in Z(u). Then the generating function 
becomes 

Z{u-x) = J2 xH - 



w>u 



Doing the same thing with z(k) and summing the resulting geometric series gives 
z(k] x) 



X" 



If t(u) 
of 



(Jl, 



1 — (x + x 2 + ■ ■ • + x k : ) 1 — 2x + x k 
l n ), then appealing to Lemma 3.1 yields a norm generating function in [n] 



Z(u: x) 



x 



2x + x n+l 



Hi 

k=l v 



X — X 



n+1 



2x + X h 



Note that this generating function depends only on the type of u and not on u itself. Note 
also that one can take n — > oo in this series (reflecting the fact that there are only finitely 
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many compositions with given norm) to obtain the norm generating function in P* 



k>l 

When u = e, this shows that the rank generating function for P* (which is graded by norm) 
is (1 — x)/(l — 2x). This can also be seen from the fact that there are 2 N ~ X compositions of 
N for N > 1. This same procedure can be applied to the generating function M(u). 

If one wants the generating function by length, then one substitutes the same variable, 
say t, for each k. Under this substitution m(k; t) = for 1 < k < n and so M(u; t) = 
unless u = e. Also, in this case one needs to remain in [«.]* since there are infinitely many 
compositions in P* of a given nonzero length. The details of these computations are routine, 
so we will merely state the results. 

Theorem 4.1. Let t(u) = (7i,...,/ n ) where u G [n]*. Then we have the norm generating 
functions 

1 — x TJ f xk ~ 

(M ' X) = l-2x + J-A 1 1 - 2x + x* 

k=l 

and 

We also have the length generating functions 



X k Yk~l+ l k 

k=2 y ' 



7 ( *\ 1 rr f (n-k + i)t 

and 

M(«;t)=-' lfU = e ' 



else. 
In P* we have norm generating functions 

Zw(u;x) = — — H 



2x - 1 - A V 1 — 2x + x k 

k>l 



and 

x \u\a_ x )2l(u)+x ! 



fc>2 v 



k\lk-l+lk 



□ 



We would now like to calculate the generating function for ( m . This is of interest because 
( m (u,w) counts the number of multichains of length m from u to w. (As mentioned in 
the introduction, the original motivation of Bergeron et. al. in studying P* was to count 
saturated chains in [e, w].) To do this, we will have to exploit a connection between the 
incidence algebra J([n]*) and the algebra EndZ(([n])) of continuous linear endomorphisms 
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of Z(([n])) (for the meaning of "continuity" here, see e.g. [2, p. 55]). This relationship will 
also be important in the next section where we will reprove the formula for \i. 

Note that (2) already defines a map Z : [n]* — > Z(([n]}). We can extend this to an 
element of EndZ(([n])) as follows. Take any <fi £ /([n]*) and define a corresponding map 
F :[n]*-Z«[n]))by 

F <t>i u ) = y^0(«,w)w 
if 

where the sum is over all w £ [n]*, or equivalently over all w; > u since <p(u, w) = otherwise. 
By continuity and linearity, we can extend to a function in EndZ(([n])) by letting 



\ It / M 



Note that the right-hand side converges since any v £ [n]* occurs with nonzero coefficient 
in only finitely many of the summands F^u). Lifting elements of /([n]*) to EndZ(([n])) in 
this way is well behaved. 

Theorem 4.2. The map <fi i— > is an algebra anti-isomorphism of I([n]*) with a subalgebra 
/EndZ«[n]». 

Proof. Checking the various needed properties of the map are easy, so we will just indicate 
why multiplication is antipreserved to illustrate. Recall that the product of (j), ifi £ I([n]*) 
is their convolution * ip while the product in End Z( ([«.])) is composition of functions. To 
show that the two multiplications correspond, it suffices to check that they do so on elements 
u £ [n]*. So we compute 

F^oF^u) = F^, ^2<p(u,v)v 

= (p(u, v)ip(v, w)w 

v,w 

= <j) * ip{u, w)w 

w 

= F^(u) 

as desired. □ 

Now we can factor the generating function for ( m as follows. Let Z[[X]] be the formal 
power series ring over the integers in the set X = {xi, X2, ■ ■ ■ , x n } of commuting variables. 
Consider the projection map p : Z(([n])) — > Z[[X]] which sends k to Xk- Then we have 

p o z{k) 



1-Xx 2 fc _i 

Define a multiplicative function / : Z[[_X]] — >• Z[[X]] by 



fM = : Xk + '-- + Xn . (4) 
1 - Xt 
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Clearly / is constructed so that 

p o z = f o p. 

We now apply the same idea to the function Z. If u G [n\* then we let X u = Ylk^k' 
where t{u) — (li, . . . ,l n ). So p{u) = X u . Define a continuous, linear map F : Z[[X]] — > Z[[X]] 
by 

F(X U ) = — i — /(I"). 

1 Xi ■ x n 

It follows that 

p o Z = F o p. 

From Theorem 4.2 we have that 

So letting t(-u) = (h,...,l n ) and applying p to both sides, we see that the generating function 
for C m in Z[[X]] is 



= F m op(n) 
= F m {X u ) 

m—l 



n .-w 



i=0 



where the last equality follows from an easy induction on m. 

Thus to find ( m for all m, it suffices to find f m {xk) for all m and fc. Since this turns out 
to be surprisingly hard to do, we will just consider what happens when n = 2. This case 
is of independent interest because then the poset has rank numbers given by the Fibonacci 
sequence. However, this is different from the Fibonacci posets defined by Stanley [12, 13]. 

For simplicity when n — 2, let x — X\ and y = x-i. In this case (4) becomes 

f(x) =x + y and f(y) 



1 — x 

To simplify notation again, let 

a m = f m (x) and b m = f m (y). 

Now we have, for m > 1. 

f m {x) = f m -\f(x)) = f m -\x + y) = r-\x) + f m -\y) 

or 

(5) 

Similarly, one obtains 

b m = (6) 

J- — a m-l 
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for m > 1, and it is easy to see that 

a = x and bo = y. 
Hence we have to solve two recurrence relations in two unknowns. 



(7) 



Let us first make the norm substitution y 



x 



In this case we will denote a m and b r 



by a m (x) and b m (x). To state our result, we will need the round-down function |_-J and 
round-up function [•] . We will also use the conventions that the binomial coefficient Q) 
equals for k < or k > n and equals 1 for k = and any n. 



Theorem 4.3. Suppose u £ [2]* has type t(u) = (hjh)- Then 



[u, w )x 



m—l 

w] =a m (x) h b m (x) l >][- -— - 



bi(x)' 



Furthermore, for all m > we have 



[X 



and b m (x) 



x 



d m (x)d m+ i(x) 



where 



a m (x) 



^(-l)UJ A m 2 + ^V* and dm {x) = ^(-1)^1 A m+ 2 'J 



x\ 



(9) 



Proof. It suffices to show that the equations for a m (x) and b m (x) given in the statement of 
the theorem satisfy (5), (6), and (7). Checking the boundary conditions is easy. 

To prove that (5) holds, substitute (8) into the recursion, multiply by d m (x)d m+ \(x)/x, 
substitute (9), and take the coefficient of x k on both sides. Thus we need to prove 



m+k— i— 2 
2 

k — i 



m+fc— i— 1 
2 

k — i 



(10) 



for k ^ 1. (When fc = lwe need to add a 1 onto the right-hand side corresponding to the x 
obtained from b m (x) after doing the multiplication. But this identity is easy to verify.) The 
proof now breaks down into four cases depending on the parities of m and k. We will only 
discuss what happens when m is even and k odd, as the other demonstrations are similar. 

So suppose m = 21 and k = 2j+l for integers I, j. Then the terms in (10) corresponding to 
even i cancel. Rewriting the odd % terms using rising factorials yields, after some cancellation, 
the equivalent hypergeometric series identity 



(/-l) i+1 (2-/)i 4^3 

= (i)i+l(l-0i 4^3 



l + l. 



-I, 
l+j-h 



-J, 

1-1-3 



-3 ~ 1/2; 
1/2 



1 



1 - I, -J, 
I - J, -I ~ J 



-J ~ 1/2; 
1/2 
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Using the implementation of Zeilberger's algorithm [15, 16] due to Paule and Schorn [8], one 
can verify that both sides of this equation satisfy the same three-term recurrence relation 
in I. Also, j ^ since k ^ 1. For positive j both sides of the equation are clearly zero 
for / = 0, 1. So since both sides also satisfy the same boundary conditions, they must be 
equal. Also, Dennis Stanton has pointed out that one can give a more traditional proof of 
this identity (and, in fact, prove a generalization of it) using Tchebyshev polynomials and 
trigonometric identities. 

Verifying (6) turns out to be much simpler. Substituting (8), clearing denominators, and 
dividing by x 2 , leads to the equivalent identity 

xa m _i(x) + d m+l (x) - d m _i(x) = 0. 

This follows easily from (9) and the binomial recursion. □ 



To get the corresponding length generating functions, we need only change the boundary 
conditions to x — y — t. In this case we write a m (t) and b m {t) for a m and b m . Since the 
computations are similar, we will simply state the result. 

Theorem 4.4. Suppose u £ [2]* has type t{u) = (h,h)- Then 

m—l ^ 



i=0 



Furthermore, for all m > we have 



a m (t) = ta , m< f! and b m (t) 
d m (t) 



d m (t)d m+ i(t) 



where a m (t) = YlA~^) 1 ' a m,i't l and d m {t) = J2i(~^-Y^rn,it z with the coefficients a m ^ and 5 n 
being given by 

( (m + l)2 i 



a, 



2i + l 



2 

m—2i 



m+2i+l 



m—2i—\ 



and 



( 



m + 2i 



m—2i 



0~m..i * 



m+2i-l 



m—2i—\ 



if 'm is even, 
if m is odd, 

if m is even, 
if m is odd. 



□ 



5 Reproving the formula for \i in P* 

We will now reprove the formula for /i in Theorem 2.2. Our principal tools will be the 
descriptions of Z and M in Lemma 3.1 and the anti-isomorphism in Theorem 4.2. Although 
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we only stated the latter result for [n]*, it clearly holds also for P*. The Lemma must be 
modified slightly by letting n tend to oo. So the formulas for z and Z become 

z(k) = [fc,oo)[fc-l]* 

and 

Z(u) = ¥*z{u). 

Proof (of Theorem 2.2). We wish to show that (*fi is the identity element of the incidence 
algebra. So by Theorem 4.2, it suffices to show that MoZis the identity endomorphism. 
For any u G P* we have, using the multiplicativity of m, 

M o Z(u) = M(¥*z(u)) = (e - l)m(F)*m(z(u)). 

So it will be enough to show 

(e - l)m(P)* = e and m(z(k)) = k 

for all ke¥. 

For the first equation, note that 

m(P) = m(l) +m(2) + m(3) H 

= (1 - 2+( e - 1)) + (2+(e - 1) - 3+(e - 1)) + (3 + ( e - 1) - 4 + ( e - 1)) + ■ • • 
= 1. 

Now (1) gives 

(e - l)m(P)* = (e - 1)1* = (e - l)(e - l)" 1 = e. 

For the second equation, we note that the case k = 1 has already been done in the 
previous paragraph, since 

m(z(l)) = m(P) = 1. 
Note that for k > 2 the same telescoping phenomenon gives 

m([fc,oo)) = fc + (e - 1) and m([k - 1]) = 1 - fc + (e - 1). 

Combining this with (1), we obtain 

m(z(k)) = m([k, oo))m([k — 1])* 

= k + (e-l) (l-fc + (e-l))* 

= fe+(e-l) (e-l + fc + (e-l))~ 1 

= fc + (e-l)(( e + fc + )( e -l))" 1 
= fe+(e- l)(e-l)- 1 (e + fc + )~ 1 
= kk^k*)- 1 
= k. 

This finishes the proof of Theorem 2.2. □ 
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6 Generalized subword order 



We now present a rubric due to Sagan and Vatter [9] under which the theorems about 
rationality of the Mobius and zeta functions for A* and P* both become special cases. Let 
P be any poset. Turn P* into a poset by letting u < p * w if there is a subword io(z'i) . . . w(ii) 
of w having length / = £(u) such that 

u{i) <p w{ii) for 1 < i < I. 

We call this the generalized subword order on P*. Note that we recover A* or P* if we take P 
to be an antichain or a well-ordered countably infinite chain, respectively. Note also that we 
will leave off the subscripts on inequalities if it is clear from context which poset is meant. 

Many of our results about £ for P* from Sections 3 and 4, as well as the corresponding 
ones for A* of Bjorner and Reutenauer [2], generalize easily to P*. Given an element a G P 
we consider the upper order ideal generated by a and its set-theoretic complement 

I a = {c G P | c > P a} and J a = P - I a , 

respectively. We define Z[u) in P* by (2) as before and also define a multiplicative map 
from P* to Z((P)) by 

z(a) = I a J*. 

The proofs we have already seen contain all the ideas needed to demonstrate the next result, 
so we suppress the details. We will also use the same notation as in the earlier results, as 
we did with Z(u). 

Theorem 6.1. Let P be any poset. Then for any u G P* we have 

Z{u) = P*z(u) 
and so Z{u) is rational. Similarly, in Z((P 2 )) the series 

z® = (( u i w ) u ® w 

u,w 

is rational. Finally, if P is finite and u has l a occurrences of a for each a G P, then we have 
the length generating function 

Generalizing our results about [i is more delicate. Indeed, there is no known formula for 
the Mobius function in P* for arbitrary P. However, there is a class of posets for which [i has 
been found. To characterize the Mobius function in these posets, we need the appropriate 
definition of a normal embedding. Suppose is a new element not in P and form a poset P 
on P U by adding the relations <p a for all a G P. One defines support and expansion 
exactly as before, just replacing with 0. Then for u, w G P*, an embedding of u into w is 
an expansion rj u G P* of length £(w) such that 

Vu{i) <p for 1 < i < £(w). 




1(3 



Clearly, u <p* w if and only if there is an embedding of u into w. 

To define normality, call P a rooted tree if its Hasse diagram is a tree having a unique 
minimal element. More generally, call P a rooted forest if the connected components of its 
Hasse diagram are rooted trees. Note that in this case P is a rooted tree. So given a G P we 
can define a~ to be the element adjacent to a on the unique path from a to in P. If P is 
a rooted forest, define an embedding t] u of u into w to be normal if it satisfies the following 
pair of conditions. 

1. For 1 < i < £(w) we have rj u {i) = w(i), w{i)~ , or 0. 

2. For all a G P and every run [r, t] of a's in w;, we have 

(a) (r, t] C Supp r] u if a is minimal in P, 

(b) r G Supp rj u otherwise. 

In this situation, the definition of the defect of a normal embedding 7] u into w should 
come as no surprise: 

d(r)u) = #{i I r)u(i) = 
The following theorem generalizes both Theorem 2.1 and Theorem 2.2. 

Theorem 6.2 (Sagan and Vatter [9]). Let P be a rooted forest. Then the Mobius function 
of P* is given by 

M^) = ]T(-ir, 

where the sum is over all normal embeddings 7] u of u into w. □ 

With this result in hand, generalizing the results for /i follows the same lines as for (. If P 
is any poset then let Op be the set of minimal elements of P. (So if P = P then Op = {1}.) 
Also, if a G P then the set of elements covering a is 

C a = {c G P | c > a and there is no b with c > b > a}. 

Now let P be a rooted forest and define M{u) for u G P* by equation (3). The corresponding 
multiplicative function is 

-Op) if a G Op, 



ma 




Op) else. 



Again, there is nothing really new in considering an arbitrary rooted forest instead of P, 
so we will merely state the results. 
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Theorem 6.3. Let P be a rooted forest. Then for any u G P* we have 



M(u) = (e - P )z{u) 
and so M(u) is rational. Similarly, in Z((P 2 )) the series 

Mg, = fi(u, w)u (g> w 

u,w 

is rational. Finally, if P is finite then we have the length generating function 

M{u-t)= { '_ P |( p| J] (l-f-|C a |(l-|0 P |f)) U(l-\C b \). 

^ ' aeOp b^Op 

In "particular, if u contains any element which is covered by exactly one other element then 
M(u;t) = 0. □ 

As a final remark, one can give a proof of Theorem 6.2 in the same way as was done for 
Theorem 2.2 in the previous section. 



7 Comments and open problems 

We end with some comments and open problems. 

7.1 Generating functions for Q 11 

It would be interesting to compute the generating function for ( m in [n]* for arbitrary n. 
It appears that one can say something, at least for n = 3. Let a m (t),b m (t),c m (t) stand 
for f m (xi), f m (x2), f m (xz), respectively, when using the length generating function. Then 
numerical evidence suggests that there is a polynomial d m (t) such that the denominators 
of our three rational functions factor as d±d2 ■ ■ ■ G?2m-2, ^2m-3^2m-2^2m-i, and G?2m-2^2m, 
respectively. Note that the behaviour of the denominator of a m (t) behaves differently from 
the n = 2 case in that the number of factors increases with m. 

It would also be interesting to find "classical" proofs of the hypergeometric identities used 
in the demonstrations of Theorems 4.3 and 4.4. The series involved are neither O-balanced 
nor well-poised so we were unable to come up with appropriate theorems in the literature 
which applied to them. Andrew Sills has noted that they are 1-balanced, which may be of 
help. 

7.2 The poset A 

Can anything be said about the Mobius function of P* if P is not a rooted forest? Again, 
computer evidence suggests that the answer is "yes." Consider the poset A in Figure 3 which 
is the smallest one to which Theorem 6.2 does not apply. Let T n (x) denote the Tchebyshev 
polynomials of the first kind, which can be defined as the unique polynomials such that 

T n (cos#) = cos(n#). 
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a b 



Figure 3: The Hasse diagram for the poset A 



Conjecture 7.1 (Sagan-Vatter [9]). For alii < j, //(a l ,e J ) is the coefficient of 1 in 



Finding a proof of this conjecture by using generating functions or any other means would 
be most welcome. 

Acknowledgment. We are indebted to Mihai Ciucu and Andrew Sills for useful discussions 
about hypergeometric series. 
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